Papers with social intelligence
Social Intelligence in the Age of LLMs (2025.naacl-tutorial)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are a powerful tool for integrating human-like communication and context-aware interactions into artificial systems. |
| Approach: | They propose to introduce and overview different aspects of artificial social intelligence and their relationship with LLMs by introducing scientific methods for evaluating social intelligence in LLM. |
| Outcome: | This tutorial will introduce scientific methods for evaluating social intelligence in LLMs, highlighting the key challenges, and identifying promising research directions. |
Social Intelligence Data Infrastructure: Structuring the Present and Navigating the Future (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing work on social intelligence in NLP does not provide a coherent subfield for researchers to analyze and identify research gaps and future directions. |
| Approach: | They build a social AI taxonomy and a data library of 480 NLP datasets to analyze existing datasets and evaluate language models’ performance in different social intelligence aspects. |
| Outcome: | The proposed infrastructure analyzes existing dataset efforts and evaluates language models’ performance in different social intelligence aspects. |
DeSIQ: Towards an Unbiased, Challenging Benchmark for Social Intelligence Understanding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Social intelligence is essential for understanding and reasoning about human expressions, intents and interactions. |
| Approach: | They propose a methodology to study the soundness of Social-IQ by applying simple perturbations to a dataset of multiple choice questions on videos of complex social interactions. |
| Outcome: | The proposed method reduces biases in the original dataset and improves performance. |
Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs (2022.emnlp-main)
Copied to clipboard
| Challenge: | We show that one of today’s largest language models lacks this kind of social intelligence out-of-the-box, using two tasks: SocialIQa and ToMi. |
| Approach: | They propose to use social intelligence and Theory of Mind to examine whether modern large-scale language models lack this kind of social intelligence out-of-the-box. |
| Outcome: | The proposed model lacks social intelligence out-of-the-box, and has well-below human accuracies on SocialIQa and ToMi, respectively. |
Spontaneous Giving and Calculated Greed in Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models demonstrate strong problem-solving abilities through reasoning techniques such as chain-of-thought prompting and reflection. |
| Approach: | They evaluate off-the-shelf reasoning models in a public goods game and compare them with and without explicit reasoning mechanisms. |
| Outcome: | The proposed models reduce cooperation and norm enforcement, favoring individual rationality, and lower collective gains in repeated interactions. |
InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Context (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated the potential to mimic human social intelligence, but most studies focus on static self-report or performance-based tests. |
| Approach: | They propose a framework to assess LLMs' ability to understand and manage intentions by mapping their ability to infer the intentions of others in a game setting. |
| Outcome: | The proposed framework assesses LLMs' ability to understand and manage intentions in a game setting. |
CoSToM: Causal-oriented Steering for Intrinsic Theory-of-Mind Alignment in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models lack intrinsic cognition and cannot generalize to complex task-specific scenarios. |
| Approach: | They propose a framework that transitions from mechanistic interpretation to active intervention to map internal distributions of ToM features and implement it via targeted activation steering within ToM-critical layers. |
| Outcome: | The proposed framework significantly enhances human-like social reasoning capabilities and dialogue quality. |
Explainable Depression Detection in Clinical Interviews with Personalized Retrieval-Augmented Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing systems rely on black-box neural networks, which lack interpretability, which is crucial in mental health contexts. |
| Approach: | They propose a Retrieval-augmented generation framework for Explainable depression detection that retrieves evidence from clinical interview transcripts, providing explanations for predictions. |
| Outcome: | The proposed framework retrieves evidence from clinical interview transcripts, providing explanations for predictions. |
SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agents (2024.acl-long)
Copied to clipboard
Ruiyi Wang, Haofei Yu, Wenxin Zhang, Zhengyang Qi, Maarten Sap, Yonatan Bisk, Graham Neubig, Hao Zhu
| Challenge: | Existing studies on building language agents have not addressed this social learning gap. |
| Approach: | They propose an interactive learning method that improves the social intelligence of language agents by using behavior cloning and self-reinforcement based training on filtered social interaction data. |
| Outcome: | The proposed method allows a 7B LLM to reach the social goal completion ability of an expert model (GPT-4-based agent) without the loss of more generic abilities, such as the ability to answer knowledge-based questions. |
Humor in Pixels: Benchmarking Large Multimodal Models Understanding of Online Comics (2025.findings-emnlp)
Copied to clipboard
| Challenge: | PixelHumor is a benchmark dataset of 2,800 annotated multi-panel comics designed to evaluate LMMs’ ability to interpret multimodal humor and recognize narrative sequences. |
| Approach: | PixelHumor is a benchmark dataset of 2,800 annotated multi-panel comics designed to evaluate LMMs’ ability to interpret multimodal humor and recognize narrative sequences. |
| Outcome: | Experiments with state-of-the-art LMMs reveal that top models achieve only 61% accuracy in panel sequencing, far below human performance. |
Minding Language Models’ (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker (2023.acl-long)
Copied to clipboard
| Challenge: | Empirical results show plug-and-play approach to reason about belief states of multiple characters in reading comprehension tasks is more precise and interpretable than previous approaches. |
| Approach: | They propose a plug-and-play approach to reason about the belief states of multiple characters in reading comprehension tasks via explicit symbolic representation. |
| Outcome: | The proposed algorithm improves theory of mind of off-the-shelf neural language models without supervision. |